<html>
<head><meta charset="utf-8"><title>understanding crater results · t-compiler/help · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/index.html">t-compiler/help</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/understanding.20crater.20results.html">understanding crater results</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="217956269"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/understanding%20crater%20results/near/217956269" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mark-i-m <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/understanding.20crater.20results.html#217956269">(Nov 26 2020 at 01:02)</a>:</h4>
<p>Can someone explain how to read crater results? For example, in <a href="https://github.com/rust-lang/rust/pull/79073#issuecomment-733986049">https://github.com/rust-lang/rust/pull/79073#issuecomment-733986049</a> lcnr says there are no regressions, but the report seems to indicate 6 regressions? I've seen this sort of thing happen in a few other cases. What am I missing?</p>



<a name="217956542"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/understanding%20crater%20results/near/217956542" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/understanding.20crater.20results.html#217956542">(Nov 26 2020 at 01:07)</a>:</h4>
<p>The regressions in the report are spurious - e.g. network error</p>



<a name="218010693"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/understanding%20crater%20results/near/218010693" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lcnr <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/understanding.20crater.20results.html#218010693">(Nov 26 2020 at 14:59)</a>:</h4>
<p>yeah, crater tries to filter failed builds by if they are spurious or not</p>



<a name="218010763"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/understanding%20crater%20results/near/218010763" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lcnr <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/understanding.20crater.20results.html#218010763">(Nov 26 2020 at 14:59)</a>:</h4>
<p>so i personally tend to look at both spurious regressed and regressed builds to see if anything looks related</p>



<a name="218010794"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/182449-t-compiler/help/topic/understanding%20crater%20results/near/218010794" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lcnr <a href="https://rust-lang.github.io/zulip_archive/stream/182449-t-compiler/help/topic/understanding.20crater.20results.html#218010794">(Nov 26 2020 at 14:59)</a>:</h4>
<p>to what was actually changed by the PR</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>